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SUMMARY 


Problem 


The  current  Naval  Reserve  Officers  Training  Corps  (NROTC)  selection  system  has  been 
helpful  in  predicting  three  criteria:  college  grade  point  average  (GPA),  grade  point  average  in  naval 
science  courses  (NSG),  and  performance  on  nonacademic  military  aspects  of  the  NROTC  program 
(APT).  In  recent  years,  the  Navy’s  desire  to  increase  the  number  of  individuals  who  giaduate  with 
engineering  or  science  degrees  has  resulted  in  a  need  to  modify  the  NROTC  selection  system  so 
that  a  fourth  criterion  can  also  be  predicted.  The  additional  criterion  is  an  applicant’s  likelihood  of 
selecting  a  technical  major  (TECH). 

Objective 

The  purpose  of  the  present  research  was  to  develop  a  new  selection  system  that  would 
simultaneously  consider  the  four  criteria.  Comparisons  among  new/experimental  and  previously 
developed  selection  composites  were  performed  to  (1)  understand  how  adding  the  fourth  criterion 
impacted  the  usefulness  of  the  selection  system  when  predicting  each  of  the  four  criteria 
individually  and  (2)  identify  the  optimum  selection  system— to  be  named  the  1989  Quality  Index 
(QI-89). 

Approach 

Selection  composites  were  developed  through  a  multi-step  process.  Initially,  a  separate 
multiple  regression  equation  was  computed  to  predict  each  of  the  four  criteria.  In  each  case,  the 
equation  contained  six  predictors:  Scholastic  Aptitude  Test  Mathematics  and  Verbal  (SATM  and 
SATV,  respectively)  scores,  peer-relative  high  school  performance  (HSR),  an  interviewer’s  ratings 
(INTER),  vocational  interest  scores  related  to  whether  an  applicant  would  remain  in  the  Navy  at 
least  1  year  past  the  minimum  obligated  service  (SCII),  scores  from  a  biodata  scale  that  predicted 
who  would  complete  the  NROTC  program  (BQ),  and  scores  on  an  interest  scale  that  was  designed 
to  identify  applicants  with  engineering  and  science  interests  (ES).  Next,  the  four  weights  for  each 
predictor  were  mathematically  combined  to  derive  a  single  weight  for  each  predictor.  Finally,  the 
predictor  weights  were  combined  to  construct  new/experimental  selection  composites  that  would 
be  predictive  of  multiple  criteria.  Other  selection  composites  were  constructed  using  the  weights 
derived  in  previous  research. 

Results  and  Conclusions 


1 .  Most  of  the  multiple-criterion  predictor  composites  resulted  in  cross-validities  of  a  similar 
magnitude  when  GPA  and  NSG  were  predicted.  When  TECH  was  predicted,  the  cross-validities 
for  the  predictor  composites  varied  a  great  deal.  As  expected,  the  new/experimental  composites 
that  included  TECH  as  a  criterion  were  more  valid  for  predicting  TECH  than  were  the  previously 
developed  predictor  composites  without  TECH.  All  composites  (including  the  composites 


specifically  designed  to  predict  ATT)  resulted  in  low  ocss-voliditief 


when  APT  was  prrdicteci. 
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2.  Regardless  of  whether  SATM  and  SATV  were  weighted  optimally,  equally,  or  in  a  ratio  of 
70  percent  to  30  percent,  the  cross-validities  were  similar  when  predicting  GPA  and  NSG. 
Composites  that  weighted  SATM  and  SATV  either  optimally  or  70  percent  to  30  percent  were, 
however,  more  valid  for  predicting  TECH.  The  latter  findings  were  expected  since  SATM 
predicted  TECH  better  than  did  SATV. 

3.  Excluding  the  BQ  from  one  of  the  new  experimental  composites  had  little  impact  on  the 
cross-validities  fcr  any  criterion. 

4.  Overall,  the  QI-89  selection  system  predicted  GPA,  NSG,  and  APT  as  well  as  or  slightly 
better  than  the  other  predictor  composites.  The  QI-89  can  also  be  used  in  conjunction  with  the  ES 
interest  measure  to  influence  the  proportion  of  NROTC  scholarship  recipients  choosing  technical 
majors. 

Recommendations 

1.  Chief  of  Naval  Education  and  Training  (CNET)  should  use  the  QI-89  and  the  ES  scale  as 
aids  in  selecting  future  NROTC  scholarship  recipients. 

2.  CNET  should  reevaluate  the  QI-89  as  component  predictors  are  modified  and  improved. 
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INTRODUCTION 


Background 

Each  year,  the  Naval  Reserve  Officer  Training  Corps  (NROTC)  receives  approximately  45,000 
inquiries  regarding  its  scholarship  program.  Nearly  12,000  of  the  individuals  contacting  NROTC 
for  information  follow  up  their  initial  requests  with  applications  for  one  of  approximately  1,500 
four-year  college  scholarships.  The  scholarships  provide  selectees  with  reimbursement  for  tuition, 
textbooks,  and  instructional  fees.  A  subsistence  allowance  of  $100  per  month  (for  a  maximum  of 
40  months)  and  pay  for  summer  training  periods  are  also  provided  to  all  persons  awarded  NROTC 
scholarships.  A  selectee,  if  accepted  by  the  college,  becomes  a  member  of  one  of  the  67  NROTC 
units  that  service  192  colleges  and  universities  located  nationwide.  Upon  graduation  from  the 
NROTC  program  and  the  college/university,  scholarship  recipients  are  commissioned  as  ensigns 
in  the  U.S.  Navy  and  must  serve  a  minimum  of  4  years  on  active  duty. 

When  determining  who  will  receive  NROTC  scholarships,  selection  boards  must  synthesize  a 
great  deal  of  applicant  information.  To  qualify  for  an  NROTC  scholarship,  an  applicant  must  have 
scored  well  on  a  series  of  variables  that  are  used  to  predict  a  variety  of  performance  criteria:  college 
grade  point  average  (GPA),  naval  aptitude  grades  (APT),  and  naval  science  grades  (NSG). 
Especially  important  to  this  study  are  six  academic  and  personal  factors:  Scholastic  Aptitude  Test- 
Verbal  (SATV),  Scholastic  Aptitude  Test-Math  (SATM),  high  school  class  rating  (HSR),  an 
interviewer’s  ratings  (INTER),  the  career-tenure  scale  developed  from  the  Strong-Campbell 
Interest  Inventory  (SCII),  and  an  NROTC-tenure  scale  developed  from  the  Background 
Questionnaire  (BQ). 

In  1979,  Neumann  and  Abrahams  (1979)  combined  these  variables  to  produce  a  composite 
called  the  Overall  Index  (OI).  To  compute  that  index,  percentage  points  were  assigned  to  each 
predictor.  SATM  and  SATV  together  received  20  percent  of  the  weight  in  the  OI;  HSR,  40  percent; 
INTER,  10  percent;  and  SCII  and  BQ  together,  30  percent.  Although  the  OI  was  helpful  in 
predicting  several  performance  criteria,  Mattson,  Neumann,  and  Abrahams  (1986)  sought 
improvements  by  developing  a  regression-based  selection  composite.  Their  composite  uses 
multiple  regression  techniques  to  optimally  weight  both  academic  and  personal  variables  to  predict 
several  performance  criteria  for  individuals  entering  NROTC  in  1979  and  1980.  As  a  result  of  that 
study,  the  1987  Quality  Index  (QI-87)  was  adopted.  The  QI-87  optimally  weighted  five  factors:  the 
sum  of  SATV  and  SATM,  HSR,  INTER,  SCII,  and  BQ  and  resulted  in  an  increased  ability  to 
predict  all  the  performance  criteria  over  that  of  the  OI. 

Problem 

The  current  NROTC  selection  system,  in  use  since  1987,  has  been  helpful  in  predicting  several 
performance  criteria.  In  recent  years,  however,  the  Navy’s  desire  to  increase  the  number  of 
individuals  graduating  with  a  technical  degree  (i.e.,  engineering  or  science)  has  made  revisions  to 
the  current  selection  composite  necessary.  In  the  past,  the  selection  composite  has  been  used 
largely  to  predict  academic  (e.g.,  grade  point  average),  and  military  performance  criteria  (e.g., 
naval  aptitude  grades).  The  current  composite  does  not  estimate  an  individual’s  likelihood  of 
selecting  a  technical  major. 


1 


Objective 


The  purpose  of  the  present  research  was  to  develop  a  new  selection  composite  that  would 
simultaneously  predict  academic  and  military  performance  as  well  as  technical  major.  To 
accomplish  this  goal,  several  new  predictor  composites  that  were  designed  to  predict  all  criteria 
were  developed.  Comparisons  were  made  among  these  composites  and  the  previous  NROTC 
selection  composite  to  identify  an  optimum  selection  system. 

APPROACH 


Sample 

The  sample  consisted  of  all  6,609  individuals  who  had  entered  NROTC  from  1983  to  1987  and 
completed  at  least  one  semester/quarter  of  the  program.  Men  comprised  96.5  percent  of  the  sample, 
and  92.6  percent  of  the  candidates  were  nonminorities.  All  of  the  individuals  included  in.  the 
sample  had  received  4- year  NROTC  scholarships  (versus  1  -year,  2-year,  or  3-year  scholarships); 
had  complete  data  on  GPA,  AFT,  NSG,  and  all  seven  predictors;  were  Navy  (versus  Marine) 
option;  and  had  a  selection  code  of  principal  selectee,  early  select,  alternate  best,  or  alternate 
middle. 

Predictors 

Seven  predictors  were  used  to  develop  the  selection  composites. 

SATV  or  American  College  Test  (ACT)  Verbal-equivalent 

This  score  represents  the  verbal  aptitude  of  the  individual  as  measured  by  a  standardized  test 
designed  for  college  admissions  and  scholarship  awards.  If  an  individual  took  the  test(s)  on 
multiple  occasions,  the  highest  score  was  used  in  the  analyses.  ACT  verbal  scores  were  translated 
to  equivalent  SATV  scores  using  a  recently  developed  conversion  table  (Owens-Kurtz,  Borman, 
Gialluca,  Abrahams,  &  Mattson,  1989). 

SATM  or  ACT  Math-equivalent 

SATM  parallels  SATV  except  that  it  represents  quantitative  aptitude. 

HSR 

This  variable  is  derived  via  a  two-step  procedure.  First,  each  individual’s  percentile  rank  is 
determined  using  the  following  formula: 

Percentile  Rank  =  (Rank  in  Class  x  2)  -  1 

Class  Size  x  2 

Second,  the  resulting  percentile  rank  is  converted  to  an  equivalent  HSR  using  the  conversion  table 
(Navy  Recruiting  Command,  11-31.2, 1988)  shown  in  Table  1 .  This  second  step  is  needed  to  lessen 
the  effect  of  the  negatively  skewed  distribution  of  percentile  ranks  (i.e.,  most  of  the  students 
graduated  in  the  upper  percentiles  of  their  high  schools).  HSR  values  can  range  from  0  to  100  in 
increments  of  10. 
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Table  1 


High  School  Record  (HSR)  Ratings  and  Equivalent  Percentiles 


HSR 

Equivalent 
Percentile  Rank 

100 

99  and  up 

90 

97-98 

80 

95-96 

70 

89-94 

60 

81-88 

50 

70-80 

40 

56-69 

30 

42-55 

20 

26-41 

10 

10-25 

0 

9  and  below 

INTER 

During  15-minute  interviews,  naval  officers  rate  the  applicants  on  several  factors  important  to 
a  career  as  a  naval  officer  (e.g.,  appearance,  poise,  oral  communication,  leadership  potential, 
motivation,  and  the  officer's  willingness  to  have  the  individual  serve  under  his/her  command). 
Based  on  the  officers’  ratings,  individuals  are  assigned  an  overall  rating  of  very  high  (1)  to  very 
poor  (5).  To  stay  consistent  with  other  variables,  this  scale  was  reverse  scored  before  it  was 
weighted  for  inclusion  in  a  composite. 

sen 

This  scale  was  designed  to  predict  officer  retention  for  at  least  1  year  beyond  an  individual’s 
minimum  obligated  service.  It  consists  of  76  item  responses  from  the  SC1I  (Campbell  &  Hansen, 
1981).  Neumann  and  Abrahams  (1978a)  reported  a  biserial  correlation  of  .25  between  the  SCII 
career  tenure  scale  and  extended  service.  Scores  on  this  scale  can  range  from  62  to  138. 

BQ 

This  scale  was  developed  in  1981  to  predict  completion  of  the  NROTC  program.  It  consists  of 
14  biographical  information  and  personality  items  that  were  selected  from  Rimland's  Background 
Questionnaire  (Neumann,  Githens,  &  Abrahams,  1967;  Rimland,  1957).  Neumann  reported  a 
biserial  correlation  of  .  12  between  the  BQ  career  tenure  scale  and  NROTC  attrition.  Scores  on  this 
scale  can  range  from  93  to  107. 
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ES 


This  scale  uses  132  item  responses  from  the  Strong-Campbell  Interest  Inventory  (Campbell  & 
Hansen,  1981)  to  identify  NROTC  applicants  with  engineering  and  science  interests.  Neumann  and 
Abrahams  (1978b)  reported  biserial  correlations  of  .56  and  .58  between  the  engineering  and 
science  interest  scale  and  choice  of  final  major  for  two  cross-validation  samples.  Scores  on  this 
scale  can  range  from  31  to  163. 

C  riteria 

Four  performance  criteria  were  used  individually  or  in  composites  for  the  analyses. 

First-year  GPA 

This  measure  is  the  GPA  obtained  from  all  college  couiowS  that  were  taken  during  the  first 
academic  year.  Grades  for  individuals  who  left  prior  to  the  end  of  the  first  academic  year  were 
cumulated  to  the  time  the  individual  left  the  NROTC  program.  For  each  host  or  cross-enrollment 
school1  GPAs  were  converted  to  standard  scores  such  that  the  mean  equalled  50  and  the  standard 
deviation  equalled  10.  By  standardising  grades,  students  attending  schools  with  different  grading 
systems  may  be  pooled  into  a  common  sample.  Standardization  was  performed  only  for  those 
schools  with  two  or  more  NROTC  scholarship  students  attending  them.  Individuals  receiving  a 
standard  score  of  more  than  five  standard  deviations  below  the  mean  were  excluded  from  the 
analyses.  The  extreme  scores  seemed  to  have  been  data-entry  errors. 

First-year  APT 

APT  is  the  first-year’s  GPA  in  nonacademic  military  aspects  of  the  NROTC  prog.am. 
Individuals  are  assigned  a  grade  of  0  to  4.00  by  NROTC  instructors  on  each  of  approximately  20 
performance  aspects  and  personal  traits  (e.g.,  goal  setting,  speaking  ability,  writing  ability, 
personr’  behavior,  maturity,  imagination,  reliability,  initiative,  military  bearing,  and  leadership). 
The  grades  are  averaged  to  obtain  an  overall  r.  unacademic  military  aptitude  grade.  APT  is 
primarily  used  to  determine  how  well  individuals  are  adapting  to  the  Navy  and  NROTC.  A  similar 
index  used  by  the  United  States  Naval  Academy  (Neumann,  Mattson,  &  Abrahams.  1989),  has 
shown  moderate  relationships  to  later  performance  as  a  commissioned  officer.  Both  the  cumulation 
of  APT  grades  for  those  individuals  who  left  prior  to  ihe  end  of  the  first  academic  year  and  the 
standardization  of  APT  were  conducted  using  the  same  procedures  outlined  for  GPA. 

First-year  NSG 

This  measure  is  the  GPA  for  naval  science  courses  taken  during  the  first  academic  year.  These 
courses  are  Navv-relevant  academic  classes  that  include  subjects  such  as  navigation  and 
seamanship.  Students  are  required  to  take  eight  such  courses  during  college,  with  most  students 
taking  one  course  each  semester.  The  cumulation  and  standardization  of  NSGs  were  conducted 
using  the  same  procedures  oudined  for  GPA  and  APT. 

'Cross -enrollment  school  is  a  college  or  university  which  is  geographically  located  near  a  host  institution  and  uses 
that  host  institution’s  NROTC  unit. 
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TECH 


This  criterion  was  developed  by  categorizing  each  college  major  as  non -technical  (1)  (e.g., 
psychology)  or  technical  (2)  (e.g.,  aeronautics).  Appendix  A  contains  the  full  list  of  majors  and 
their  corresponding  categories  that  were  obtained  from  the  Chief  of  Naval  Education  and  Training 
(CNET). 

Individuals  who  had  TECH  scores  represented  a  subset  of  the  larger  sample.  TECH  was 
considered  valid  if  the  candidate  had  entered  college  in  (1)  1983,  1984,  or  1985  or  (2)  1986  and 
had  completed  at  least  one  semester/quarter  of  his/her  junior  year.  It  was  assumed  that  individuals 
who  had  both  entered  NROTC  in  1983-1985  and  declared  a  final  major  either  would  have 
graduated  or  be  far  enough  along  in  their  course  work  to  complete  the  degree  under  the  major  they 
had  indicated.  The  sa*ne  assumption  was  made  for  individuals  in  entering  year  1986  who  had 
started  their  junior  year  in  college. 

Procedure 

Development  and  Cross-validation  Samples 

The  sample  was  split  into  three  subsamples.  To  generate  the  first  two  samples,  the  5,957  people 
who  entered  NROTC  between  1983  and  1986  were  randomly  assigned  to  either  a  development  or 
cross-validation  (i.e.,  hold-out)  sample  (N  =  3,652  and  N  =  2,305,  respectively).  The  larger 
proportion  of  the  sample  (i.e.,  60%)  was  assigned  to  the  development  subsample  to  maximize  the 
stability  of  the  predictor  weights.  The  third  sample,  652  individuals  who  entered  NROTC  during 
1987,  was  used  as  a  second  cross-validation  sample.  The  second  cross-validation  was  performed 
to  ensure  that  weights  developed  on  earlier  classes  remained  stable  for  the  most  recent  year  for 
which  criterion  data  were  available. 

Developing  Optimally  Weighted  Composites 

Validity  coefficients  corrected  for  range  restriction  were  used  in  multiple  regression  to 
develop  an  optimally  weighted  selection  composite  for  predicting  each  of  the  four  individual 
criteria.  Although  this  procedure  results  in  four  separate  composite  scores,  to  make  selection 
decisions,  applicants  must  ultimately  be  rank-ordered  on  a  single  metric.  To  obtain  such  overall 
composites  the  single-criterion  composites  were  combined  in  two  different  ways.  One  combination 
(COMP3)  included  the  composites  designed  to  predict  GPA,  APT,  and  NSG,  and  the  other 
(COMP4)  included  composites  designed  to  predict  all  four  criteria.3  Weights  were  derived  for 
these  overall  composites  by  combining  predictor  weights  obtained  for  the  single  criterion 
regression  equations.  The  procedure  used  to  obtain  these  weights  is  outlined  in  Appendix  B. 


2Because  selectees  tend  to  have  higher  predictor  scores  than  non-selectees,  the  range  of  predictor  scores  is  re¬ 
duced.  As  a  result  of  this  reduced  variability,  a  correlation  based  on  a  restricted  sample  would  underestimate  the  rela¬ 
tionship  between  a  predictor  and  a  criterion  for  the  applicant  population. 

Representatives  of  the  Chief  of  Naval  Education  and  Training  supplied  the  criterion  weights  (See  Appendix)  for 
the  multiple  criterion  composites. 
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Each  of  the  composites  were  then  cross-validated  on  both  the  1983-1986  and  1987  hold-out 
samples.  The  composites  were  evaluated  for  their  ability  to  predict  GPA,  APT,  NSG,  and  TECH 
in  the  1983-1986  sample,  and  their  ability  to  predict  GPA,  APT,  and  NSG  in  the  1987  sample. 

Developing  Experimental  Composites 

Using  the  same  criterion  weights  employed  in  the  optimally  weighted  composites,  the  same 
procedure  was  followed  to  create  two  experimental  predictor  composites.  The  first  experimental 
predictor  composite  assessed  the  contribution  of  the  BQ  in  predicting  die  four  criteria  by 
eliminating  it  from  the  composite  and  optimally  weighting  the  remaining  six  predictors.  For  the 
second  experimental  composite,  SATM  and  SATV,  respectively,  received  70  percent  and  30 
percent  of  the  total  weight  assigned  to  these  two  variables  in  the  QI-87.  The  cross-validation  of  the 
two  experimental  composites  paralleled  that  of  the  optimally  weighted  composites. 

Using  Previously  Developed  Operational  Composites 

The  QI-87  and  the  OI  were  also  revalidated  on  the  two  hold-out  samples.  Mattson  et  al.  (1986) 
supplied  the  b  weights  that  were  used  to  compute  the  QI-87  and  the  rational  weights  that  were  used 
to  compute  the  OI. 

Determining  Effective  Weights 

To  assess  the  percentage  of  weight  that  each  predictor  received  in  the  various  composites, 
effective  weights  scaled  to  100  percent  were  computed.  Effective  weights  allow  for  comparisons 

(1)  within  each  selection  composite  to  determine  which  predictors  contributed  the  most  weight  and 

(2)  between  the  selection  composites  to  determine  if  the  contribution  of  each  predictor  changed 
depending  on  the  selection  composite.  To  compute  the  effective  weights,  the  un standardized  b 
weights  for  each  predictor  within  a  composite  were  first  multiplied  by  the  corresponding  standard 
deviation  for  that  predictor.  The  products  of  b  times  SD  were  then  summed  across  all  the  predictors 
included  in  the  composite.  Each  product  was  then  divided  by  that  sum  and  multiplied  by  100. 

Evaluation  of  Composites 

Tests  of  statistical  significance  were  not  performed  in  any  of  the  following  analyses  for  two 
reasons.  The  primary  reason  concerns  the  previously  mentioned  correction  of  validity  coefficients 
for  range  restriction.  These  corrections  were  made  on  individual  predictor-criterion  correlations 
before  they  were  used  to  construct  composites  for  both  the  developmental  and  cross-validation 
samples.  While  the  corrected  coefficients  provide  a  more  appropriate  basis  for  developing  and 
evaluating  the  composites,  standard  statistical  tests  are  not  appropriate  for  assessing  the 
significance  of  validity  coefficients  once  they  have  been  corrected. 

Second,  the  large  subsamples  may  blur  the  lines  between  statistical  and  practical  significance. 
For  example,  a  zero-order  correlation  coefficient  of  r  =  .062  for  1 ,000  subjects  is  significant  at  the 
.05  level;  however,  the  practical  value  of  a  statistically  “significant”  predictor  of  this  magnitude 
when  used  alone  may  be  questioned.  Nevertheless,  such  a  predictor  may  be  useful  under  one  or 
both  of  the  following  conditions:  (1)  when,  in  combination  with  other  predictors,  it  provides  a 
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unique  contribution  to  the  prediction  of  the  criterion  or  (2)  when  large  numbers  of  selection 
decisions  must  be  made  (as  is  done  for  NROTC  selection). 

RESULTS  AND  DISCUSSION 


Development  Sample 

Table  2  shows  the  means  and  standard  deviations  for  the  predictors  and  criteria,  and 
correlations  for  those  two  sets  of  variables.  This  information  is  provided  for  both  the  entire 
development  sample  and  the  development  subsample  that  had  valid  scores  on  the  TECH  criterion. 
After  corrections  for  range  restriction,  validities  increased  approximately  .02  to  .03. 

Academic  Potential  of  the  Average  NROTC  Applicant 

The  SATM  means  indicate  that  the  average  NROTC  scholarship  student  scored  very  high  in 
mathematics  aptitude.  A  SATM  score  of  642  corresponds  to  approximately  the  89th  percentile.  In 
other  words,  the  average  NROTC  scholarship  student  outperforms  89  percent  of  the  high  school 
seniors  who  take  the  mathematical  portion  of  the  SAT  (College  Entrance  Examination  Board, 
1989).  The  SATV  and  HSR  means  are  also  above  average.  For  SATV,  the  average  NROTC 
scholarship  student  outperforms  approximately  87  percent  of  the  college-bound  seniors  taking  that 
test  (College  Entrance  Examination  Board,  1989).  Finally,  the  average  NROTC  scholarship 
student  had  an  HSR  of  73.55.  That  HSR  value  indicates  that  the  average  NROTC  scholarship 
recipient  graduated  in  the  top  10  percent  of  his/her  high  school  class  (per  Table  1). 

Comparison  of  Development  Sample  and  Subsample  Findings 

There  are  negligible  differences  between  the  predictor  and  criterion  means  and  standard 
deviations  for  the  full  development  sample  and  its  subsample.  The  intercorrelations  among  GPA, 
APT,  and  NSG  are  slightly  lower  for  the  development  subsample  than  for  the  full  development 
sample.  The  three  criteria  were  moderately  intercorrelated,  with  NSG  and  GPA  being  the  most 
highly  related  (.562  in  the  development  sample).  This  result  would  be  expected  because  these  two 
criteria  measure  academic  aspects  of  the  NROTC  program  and  college;  furthermore,  NSG  is 
computed  from  a  subset  of  the  courses  included  in  GPA.  These  relationships  are  also  consistent 
with  the  findings  of  Mattson  et  al.  (i986).  In  that  study,  correlations  among  the  criteria  varied  from 
.40  to  .54,  and  GPA  and  NSG  were  the  most  highly  intercorrelated  of  the  three  criteria.  GPA 
showed  the  highest  relationship  with  TECH,  a  criterion  not  included  in  earlier  selection 
composites.  The  correlations  between  the  three  criteria  and  TECH  are,  however,  much  smaller  than 
the  intercorrelations  among  GPA,  APT,  and  NSG. 
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Table  2 


Descriptive  Statistics  for  the  Full  Development  Sample  and  the  Development  Subsample 


Variable 

Mean 

SD 

GPA 

Correlations  with  Criteria 
APT  NSG 

TECH 

Predictor 

SATV 

558.51 

76.78 

.124 

.027 

.192 

— 

SATV 

r 

560.08 

76.98 

.101 

.013 

.193 

-.051 

SATMf 

642.91 

64.35 

.187 

.036 

.092 

— 

SATM 

r 

642.85 

63.99 

.183 

.047 

.054 

.221 

HSR, 

73.55 

16.81 

.272 

.132 

.171 

_ 

HSR 

r 

74.45 

16.34 

.280 

.105 

.156 

.093 

INTERf 

4.82 

.48 

.035 

.093 

.019 

INTER 

r 

4.83 

.46 

.030 

.061 

.016 

-.008 

sen 

105.36 

5.% 

-.076 

-.010 

-.021 

_ 

SCII 

r 

105.49 

5.88 

-.069 

-.008 

-.014 

.066 

BQ 

100.97 

2.32 

.007 

.047 

.067 

_ 

BQr 

101.09 

2.30 

-.006 

.010 

.052 

• 

o 

s 

ES 

110.19 

13.55 

.013 

.022 

.083 

.... 

ESf 

r 

110.62 

13.28 

.024 

.033 

.094 

.399 

Criterion 

GPA 

49.88 

9.66 

1.000 

GPA 

r 

51.72 

8.16 

1.000 

APT 

49.71 

9.77 

.425 

1.000 

APT 

r 

52.15 

8.43 

.363 

1.000 

NSG- 

49.80 

9.77 

.562 

.419 

1.000 

NSG 

r 

51.66 

8.57 

.546 

.363 

1.000 

TECH- 

_ 

_ 

_ 

_ 

_ 

_ 

TECH 

r 

1.59 

.49 

.243 

.104 

.161 

1.000 

f  as  a  subscript  denotes  the  Full  Development  Sample  (N  =  3,652). 
r  as  a  subscript  denotes  the  Development  Subsample  (N  =  2,077). 


The  predictor-criterion  correlations  varied  little  in  magnitude  between  the  full  development 
sample  and  its  subsample.  For  both  groups,  HSR  was  the  variable  most  highly  correlated  with  GPA 
and  APT.  SATV  and  HSR  showed  the  strongest  relationships  with  NSG.  Although  SATM  and 
SATV  showed  strong  relationships  with  GPA  and  NSG,  respectively,  they  showed  virtually  no 
relationship  with  APT.  The  interview  rating,  however,  was  related  to  APT.  These  latter  two  sets  of 
findings  are  consistent  with  the  observation  that  GPA  and  NSG  measure  the  academic  performance 


of  NROTC  participants  while  APT  measures  military  characteristics  of  the  future  officers.  ES  was 
the  predictor  most  highly  correlated  with  TECH.  This  outcome  was  expected  because  the  ES  scale 
was  specifically  developed  to  predict  final  major.  Finally,  SATM  also  had  a  moderate  association 
with  TECH. 

Cross-validation 

Predictor  scores  were  computed  for  each  of  the  nine  composites  (i.e.,  the  four  single-criterion, 
two  multiple-criterion,  and  two  experimental  composites,  and  the  QI-87)  using  data  from  the  hold¬ 
out  samples.  These  nine  composite  scores  and  the  score  btained  on  the  OI  were  then  correlated 
with  each  criterion. 

Single-criterion  Composites 

Table  3  shows  the  cross- validity  coefficients  that  were  obtained  for  the  four  single-criterion 
composites  using  data  from  the  1983-1986  and  1987  hold-out  samples.  The  cross-validities  for  the 
single-criterion  composites  are  provided  principally  to  show  the  upper  limit  of  prediction  for  a 
given  criterion  since  each  composite  should  predict  its  own  criterion  better  than  any  of  the  other 
composites.  To  use  the  table,  the  criterion  of  interest  is  located  in  the  right-hand  columns,  and  the 
predictor  composite  is  located  in  the  left-hand  column.  The  cross-validity  is  found  at  the 
intersection  of  the  corresponding  column  and  row.  For  example,  the  .122  shown  on  the  first  row, 
second  column  of  Table  3  indicates  the  cross-validity  estimate  that  was  obtained  when  weights  that 
were  derived  to  optimally  predict  GPA  (for  the  development  sample)  were  used  to  predict  APT  in 
the  1983-1986  hold-out  sample. 


Table  3 

Cross-validity  Coefficients  for  the  Optimal,  Single-criterion  Composites 
in  the  1983-1986  and  1987  Hold-out  Samples 


Variable 

GPA 

Criteria 

APT  NSG 

TECH 

GPA  Composite 

.289 

.122 

337 

.138 

GPA^  Composite 

.304 

-.001 

.228 

c 

APT  Composite 

.219 

.137 

330 

.108 

APT^  Composite 

320 

.073 

.193 

c 

NSG  Composite 

NSG*  Composite 

320 

.118 

391 

.121 

.227 

.009 

.286 

c 

TECH  Composite 

.102 

.036 

.137 

.430 

TECH?  Composite 

.124 

.056 

.063 

c 

a  as  a  subscript  denotes  the  1983-1986  Hold-out  Sample  (N  =  2305). 
b  as  a  subscript  denotes  the  1987  Hold-out  Sample  (N  =  652). 

c  as  a  subscript  denotes  the  1983-1986  Reduced  Hold-out  Sample  (N  =  1,313)  with  a  valid  TECH  score. 
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Of  primary  interest  are  the  bold-faced  values  shown  on  the  diagonal.  These  values  reflect  the 
predictive  ability  for  each  composite’s  target  criterion;  that  is,  the  GPA  composite  reveals  a  cross¬ 
validity  of  .289  with  the  GPA  criterion.  These  diagonal  values  may  be  compared  with  the  four 
corresponding  validities  observed  for  these  composites  in  the  developmental  sample;  .327  for 
GPA;  .175  for  APT;  .297  for  NSG;  and  .455  for  TECH  (untabled).  As  expected,  these 
development- sample  validities  are,  in  general,  slightly  higher  than  the  corresponding  cross- 
validities  indicating  a  modest  amount  of  shrinkage.  The  GPA,  NSG,  APT,  and  TECH  composites 
each  predicted  its  target  criterion  better  than  any  of  the  other  composites.  Surprisingly,  the  APT 
composite  was  a  better  predictor  of  GPA  and  NSG  than  of  APT.  Overall,  three  of  the  four 
composites  (GPA,  APT,  and  NSG)  were  better  predictors  of  GPA  and  NSG  than  of  APT  and 
TECH. 

Another  important  finding  is  that  none  of  the  composites  had  a  meaningful  negative 
relationship  with  the  non-target  criteria.  Had  a  negative  relationship  been  observed  between  any  of 
the  predictor  composites  and  NROTC  performance  criteria,  the  decision  to  use  the  predictor  in  the 
selection  system  would  have  been  problematic.  For  example,  this  situation  would  have  occurred  if 
individuals  who  tend  to  score  high  on  the  GPA  composite  also  tend  to  have  low  APT  criterion 
values. 

The  coefficients  obtained  on  the  second  cross-validation  sample  are  shown  directly  under  the 
bold-faced  cross-validities.  Across  all  four  single -criterion  composites,  the  cross-validities 
obtained  on  the  1987  sample  varied  little  from  those  obtained  on  the  1983-1986  sample  when  GPA 
and  NSG  were  predicted  (see  Table  3).  All  of  the  composites  predicted  GPA  slightly  better  in  the 
1987  sample  (than  in  the  1983-1986  sample)  and  NSG  slightly  better  in  the  1983-1986  sample 
(than  in  the  1987  sample).  Somewhat  larger  differences  were  found  between  the  cross- validities 
obtained  for  the  two  samples  when  predicting  APT.  The  GPA,  APT,  and  NSG  composites 
predicted  APT  better  in  the  1983-1986  sample  than  in  the  1987  sample.  The  APT  coefficients  were, 
however,  very  low  in  both  cases.  Further,  the  cross-validities  for  the  two  samples  were  both  near 
.00  when  the  TECH-derived  composite  was  used  to  predict  APT.  Inspection  of  the  correlations 
between  APT  and  several  highly-weighted  predictors  (i.e.,  HSR,  SATM,  and  SATV)  revealed 
higher  relationships  for  the  1983-1986  sample  than  for  the  1987  sample.  Thus,  the  relatively  higher 
validities  obtained  for  the  individual  predictors  in  the  1983-1986  sample  appeared  to  account  for 
the  subsequent  differences  in  predictive  ability  for  these  composites. 

Multiple-criterion  Composites 

Table  4  shows  the  cross-validity  coefficients  for  die  three  sets  of  multiple-criterion  composites: 
optimal,  experimental,  and  operational,  that  were  obtained  using  data  from  the  1983-1986  and 
1987  hold-out  samples.  As  was  found  with  the  composites  designed  to  predict  a  single  criterion, 
very  little  difference  was  detected  between  the  cross-validities  obtained  on  the  1983-1986  and 
1987  samples  when  GPA  and  NSG  were  predicted.  Also  similar  to  the  earlier  findings  were  the 
pattern  of  differences  between  the  cross-validities  obtained  for  the  two  samples  when  predicting 
APT.  Although  all  of  the  cross-validities  for  APT  were  very  low,  the  composites  predicted  APT 
marginally  better  in  the  1983-1986  sample  th.Jt  in  the  1987  sample. 
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Table  4 


Cross-validity  Coefficients  for  the  Multiple-criterion  and  the  Operational  Composites 
in  the  1983-1986  and  1987  Hold-out  Samples 


Multiple-criterion  and 

Operational  Composites 

TIPS - 

Criteria 
- £PT - 

use 

— TECIT 

Optimal  Composites 

COMP3 

.269 

.134 

.277 

.135c 

COMP3!* 

0 

219 

.024 

.257 

— 

COMP4 

.246 

.114 

260 

292c 

COMP4* 

0 

259 

.037 

221 

— 

Experimental  Composites 

Experimental  1  (no  BQ) 

Experimental  1*  (r.o  BQ) 

.246 

.110 

253 

.297c 

.257 

.019 

.232 

— 

Experimental  2  (SATM  70%/S  ATV  30%) 
Experimental  2?  (SATM  70%/SATV  30%) 

D 

.245 

.114 

.259 

.292c 

.257 

.038 

.220 

— 

Operational  Composites 

Quality  Index  (87) 

.274 

.137 

251 

.107c 

Quality  Index^  (87) 

.278 

.024 

228 

— 

Overall  Index 

.194 

.102 

.205 

.090 

Overall  Index!" 

_ -D _ 

~d 

— d 

— d 

"d 

®As  a  subscript  denotes  the  1983-1986  Hold-out  Sample  (N  =  2,305). 
bAs  a  subscript  denotes  the  1987  Hold-out  Sample  (N  =  652). 

^As  a  subscript  denotes  the  1983-1986  Reduced  Hold-out  Sample  (N  =  1,313)  with  a  valid  TECH  score. 

“Since  the  OI  was  discontinued  after  the  1986  selection  cycle,  scores  for  this  composite  were  not  available  for  the  1987 
cross-validation  sample. 

The  majority  of  the  composites  were  better  predictors  of  GPA  and  NSG  than  of  APT  and 
TECH.  Although  the  magnitudes  of  the  cross-validities  were  slightly  higher  in  the  Mattson  et  al. 
(1986)  study  (many  of  the  validities  fell  above  .30),  the  two  studies  revealed  the  same  pattern  of 
results.  Interestingly,  three  (COMP4  and  Experimental  1  and  2)  of  the  six  composites  produced 
higher  validities  when  predicting  TECH  than  when  predicting  GPA,  APT,  or  NSG.  Considering 
the  findings  for  all  of  the  criteria,  it  appears  that  for  both  samples  the  COMP4  and  the  two 
experimental  composites  would  be  the  most  useful  for  simultaneously  predicting  all  four 
performance  criteria. 

The  similarity  between  the  validities  for  the  Experimental  1  and  the  COMP4  composites,  was 
expected  since  the  two  composites  are  the  same  except  that  the  Experimental  1  composite  does  not 
include  the  BQ.  Because  the  BQ  was  designed  to  predict  officer  tenure  and  showed  little 
relationship  to  college  criteria,  its  exclusion  had  little  impact.  On  the  other  hand,  elimination  of  the 
BQ  may  relate  to  a  length  of  service  criterion. 
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The  validities  for  the  Experimental  2  and  the  COMP4  composites  were  also  very  similar, 
showing  that  the  70  percent-30  percent  forced  weighting  of  SATM  and  SATV  scores  had  little 
effect.  This  result  is  largely  due  to  the  fact  that  the  forced  weights  for  these  variables  were  very 
close  to  the  optimal  weights  (SATM  69%  to  SATV  31%)  derived  for  the  COMP4. 

Comparisons  between  the  validities  for  the  COMP4  and  the  previously  developed  operational 
composites  indicate  that  both  the  COMP4  and  the  QI-87  are  superior  to  the  OI  in  predicting  each 
of  the  four  criteria.  The  differences  between  the  CO  MP4  composite  and  the  QI-87  in  predicting 
GPA,  APT,  and  NSG  are  slight;  whereas,  the  differences  between  these  two  composites  when 
predicting  TECH  clearly  favor  die  COMP4.  This  advantage  for  the  COMP4  is  partially  due  to  the 
fact  that  the  development  of  the  CO  MP4  included  TECH  as  a  criterion  and  is,  therefore,  more  valid 
than  the  QI-87  for  predicting  this  criterion. 

Effective  Weights 

Table  5  shows  the  effective  weights  for  each  of  the  predictors  included  in  the  different  selection 
composites.  Overall,  HSR  received  the  highest  weight  in  the  majority  of  the  composites.  SATM 
and  the  newly  added  ES  also  received  relatively  high  weights  for  several  of  the  composites. 
Comparisons  of  the  effective  weights  for  the  COMP4  with  those  of  the  two  operational  composites 
provide  possible  explanations  for  the  differences  in  cross-validities  among  these  composites  for 
predicting  each  of  the  criteria.  One  reason  concerns  the  addition  of  ES  as  a  predictor  in  the  COMP4. 
Because  ES  is  highly  correlated  with  TECH  and  received  a  large  weight,  it  made  the  COMP4  a 
much  stronger  predictor  of  TECH  and  somewhat  less  valid  for  predicting  GPA  and  APT.  Two 
additional  differences  are  the  COMP4’s  decreased  emphasis  on  HSR  and  its  use  of  optimal  weights 
for  SATM  and  SATV.  In  contrast,  the  QI-87  assigns  a  much  higher  weight  to  HSR  and  equal 
weights  to  SATM  and  SATV. 

In  summary,  the  COMP4  composite  assigns  optimal  weights  to  SATV  and  SATM,  includes  ES 
in  the  selection  composite,  reduces  the  weight  given  to  HSR,  and  includes  TECH  as  an  additional 
criterion.  These  differences  slightly  lowered  the  predictability  of  GPA  and  APT  (COMP4  r  = 
.246  versus  QI-87  r  =  .274)  (COMP4  r  =  .114  versus  QI-87  r  =  .137),  slightly  increased  the 
ability  to  predict  NSG  (COMP4  r  =7260  versus  QI-87  r  =  .251),  and  substantially  improved  the 
ability  to  predict  TECH  (COMPEL  r  =  .292  versus  QI-87  r  =  .107). 

Modifying  Effective  Weights 

Revision  1 

Based  on  the  results,  the  COMP4  and  the  two  experimental  composites  were  determined  to  be 
the  most  potentially  useful  prediction  systems  for  NROTC  scholarship  selection  when  attempting 
to  predict  all  four  criteria.  It  should  be  noted  that  although  the  GPA  predictor  composite  is  just  as 
effective  for  predicting  GPA,  APT,  NSG  as  the  three  composites  just  mentioned,  the  desire  to 
improve  the  prediction  of  technical  major  resulted  in  the  choice  of  COMP4  and  the  two 
experimental  composites  as  potentially  useful  selection  systems.  An  examination  of  the  effective 
weights,  however,  revealed  that  the  ES  score  received  a  higher  percentage  (approximately  24%)  of 
the  total  weight  than  CNET  desired.  The  magnitude  of  the  ES  weight  was  primarily  due  to  the  large 
criterion  weight  assigned  to  TECH  (.25)  in  the  criterion  composite.  To  indirectly  lower  the 
percentage  of  total  weight  received  by  ES,  the  weight  assigned  to  the  TECH  criterion  was  reduced 
from  .25  to  .10.  The  remaining  criteria  were  then  reweighted  as  follows:  GPA,  .40;  APT,  .30;  and 
NSG,  .20. 


12 


Table  5 


Effective  Weights  for  Predictors  in  Different  Selection  Composites 


Variable  the  Composite  was  Originally 

Predictors 

Designed  to  Predict 

SATV 

SATM 

HSR 

INTER 

scii 

BQ 

ES 

Optimal  Single-criterion  Composites 

GPA 

11 

28 

58 

4 

0 

2 

-1 

APT 

3 

2 

44 

27 

4 

13 

7 

NSG 

36 

-2 

28 

1 

9 

12 

16 

TECH 

-25 

45 

13 

2 

8 

-1 

58 

Optimal  Multiple-criterion  Composites 

COMP3 

17 

10 

42 

8 

5 

9 

9 

COMP4 

8 

18 

33 

6 

6 

5 

24 

Experimental  Composites 

Experimental  1  (no  BQ) 

9 

19 

35 

7 

6 

24 

Experimental  2  (SATM  70%/SATV  30%) 

7 

16 

35 

6 

5 

6 

25 

Operational  Composites 

Quality  Index  (87) 

11 

ii 

55 

11 

9 

3 

- 

Overall  Index 

<— 

-20  -  -> 
a 

40 

10 

<— 

A 

1 

o 

a 

k  An  effective  weight  of  20  was  applied  to  the  sum  of  S ATV  and  SATM. 
An  effective  weight  of  30  was  applied  to  the  sum  of  SCO  and  BQ. 


The  proposed  change  in  criterion  weights  required  the  development  of  new  composites. 
Predictor  weights  for  the  three  new  composites  were  calculated  in  the  same  manner  described 
previously,  using  the  new  criterion  weights.  The  predictor  weights  were  then  cross-validated  on  the 
1983-1986  and  the  1987  samples.  The  resulting  cross-validities  appear  in  Table  6.  Comparisons 
between  Table  6  and  Table  4  show  that  reducing  the  weight  of  TECH  in  the  three  composites 
resulted  in  an  increased  ability  to  predict  GPA,  APT,  and  NSG  in  the  1983-1986  sample,  and  GPA 
and  NSG  in  the  1987  sample.  These  increases  in  predictive  ability  were  accompanied  by  decreases 
in  the  ability  to  predict  TECH.  The  cross-validities  that  were  obtained  when  the  COMP4  and 
Experimental  1  and  2  composites  were  used  to  predict  TECH  with  the  initial  criterion  weights 
ranged  from  .292  to  .297;  whereas,  the  cross-validities  ranged  from  .204  to  .214  when  the  revised 
set  of  criterion  weights  were  used. 

It  was  also  necessary  to  compare  the  cross-validities  (from  Table  6)  for  the  three  revised 
composites  with  the  QI-87  cross-validities  (from  Table  4).  Although  all  four  composites  predicted 
GPA,  APT,  and  NSG  equally  well,  the  COMP4(R)  and  the  two  revised  experimental  composites 
remained  better  predictors  of  TECH  (than  did  the  QI-87),  even  with  the  reduction  in  weight 
assigned  to  TECH.  The  new  effective  weights  for  the  COMP4(R)  and  the  two  experimental  revised 
composites  appear  in  Table  7.  Tables  5  and  7  show  that  the  effective  weight  assigned  to  ES  in  each 
revised  composite  has  been  reduced  by  approximately  half.  For  example,  the  effective  weight  for 
ES  in  the  COMP4  was  24  percent.  After  the  change  in  criterion  weights,  the  effective  weight  for 
ES  was  reduced  to  13  percent. 
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Table  6 


Cross-validities  for  Revision  1  and  Revision  2 


Criteria 


Revised  Composites 

GPA 

APT 

NSG 

TECH 

Revision  1 

COMP4(R) 

271 

.128 

269 

.204 

COMP4(R)® 

283 

.027 

243 

c 

Experimental  1(R) 

271 

.123 

261 

.209 

Experimental  1(R)^ 

.282 

.016 

235 

Experimental  2(R) 

271 

.127 

.260 

.214 

Experimental  2(R)^ 

283 

.032 

232 

Revision  2 

Experimental  3  (no  ES) 

.282 

.126 

.246 

.141 

Experimental  3  (no  ES)h 

296 

.011 

.238 

a  as  a  subscript  denotes  the  1983-1986  Hold-out  Sample  (N  =  2,305). 
b  as  a  subscript  denotes  the  1987  Hold-out  Sample  (N  =  652). 

c  as  a  subscript  denotes  the  1983-1986  Reduced  Hold-out  Sample  (N  =  1,313)  with  a  valid  TECH  score. 


Table  7 


Effective  Weights  for  Revision  1  and  Revision  2 


Predictors 

Revised  Composites 

SATV 

SATM 

HSR 

INTER 

SCII 

BQ 

ES 

Revision  1 

COMP4(R) 

12 

16 

41 

8 

6 

6 

13 

Experimental  1(R) 

13 

16 

43 

9 

5 

- 

14 

Experimental  2(R) 

8 

19 

44 

8 

4 

7 

13 

Revision  2 

Experimental  3  (no  ES) 

10 

25 

47 

8 

4 

6 

- 

Revision  2 


Despite  the  large  reduction  in  weight  assigned  to  ES,  it  continued  to  receive  a  higher  than 
desired  weight.  The  CNET  representatives  were  concerned  that  applicants  with  outstanding 
credentials  might  not  receive  NROTC  scholarships  if  their  interests  tended  toward  non-technical 
fields  of  study.  To  correct  this  problem,  the  ES  was  eliminated  from  the  predictor  composite 
entirely.  To  retain  its  usefulness  for  selection,  CNET  officials  requested  that  ES  scores  be 
converted  to  a  20-point  scale.  For  this  purpose,  raw  ES  scores  were  transformed,  via  linear 
interpolation,  to  an  adjusted  scale  ranging  from  0  to  20.  This  transformation  was  accomplished  by 
the  following  equation: 

Adjusted  ES  Score  =  (Raw  ES  Score  x  .1961)  -  10.1972 

Using  the  adjusted  score  range,  an  expectancy  table  was  constructed  (on  the  development  subsam¬ 
ple)  for  use  in  interpreting  an  individual’s  ES  score.  The  expectancy  table  is  shown  in  Figure  1.  To 
use  the  table,  an  individual’s  transformed  ES  score  is  located  in  the  table,  and  the  likelihood  of  that 
individual  selecting  a  technical  final  major  can  be  determined. 

Removal  of  ES  from  the  predictor  composite  required  the  redistribution  of  its  effective  weight 
(13%)  to  the  remaining  six  predictors.  Using  the  effective  weights  from  COMP4(R),  CNET 
representatives  created  Experimental  3  (No  ES)  by  adding  9  percentage  points  to  SATM,  adding  6 
percentage  points  to  HSR,  and  reducing  SATV  by  2  percentage  points;  the  weights  assigned  to 
INTER,  SCII,  and  BQ  remained  the  same  as  in  the  COMP4(R)  composite.  The  effective  weights 
for  Experimental  3  (no  ES)  composite  appear  in  the  bottom  row  of  Table  7.  As  shown,  the  weight 
that  was  assigned  to  ES  was  redistributed  so  that  SATM  and  SATV  received  additional  weight  but 
remained  at  approximately  a  70-30  ratio.  The  computing  (i.e.,  b)  weights  for  the  six  remaining 
predictors  were  .0585  for  SATV,  .1676  for  SATM,  1 .0386  for  HSR,  4.7730  for  INTER,  .2875  for 
SCII,  and  1.1264  for  BQ. 

The  cross-validities  for  the  Experimental  3  composite  appear  in  Table  6.  As  shown,  the 
Experimental  3  composite  predicted  GPA  as  well  as  the  COMP4(R),  the  QI-87,  and  the  two 
previously  revised  experimental  composites.  The  Experimental  3  composite  also  resulted  in  cross- 
validities  for  APT  and  NSG  that  were  comparable  to  those  obtained  for  the  four  multiple-  riterion 
composites  just  mentioned.  As  a  result  of  removing  ES,  the  cross-validity  for  TECH  fell  below  that 
of  the  COMP4(R)  and  the  two  experimental  composites.  The  Experimental  3  composite  did, 
however,  remain  a  better  predictor  of  TECH  than  was  the  QI-87  (see  Table  4). 
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PRACTICAL  IMPLICATIONS 


Figure  2  illustrates  the  practical  impact  of  replacing  the  QI-87  with  the  Experimental  3 
composite,  hereafter  referred  to  as  the  QI-89.  Figures  2a  through  2c  show  the  expected  percentage 
of  midshipmen  who  will  earn  above  average  GPAs,  APTs,  and  NSGs,  respectively,  after  1  year  of 
college.  These  percentages  are  based  on  the  cross- validities  obtained  on  the  1983-1986  hold-out 
sample.  The  figures  suggest  that  the  percentage  of  midshipmen  earning  above  average  grades  is 
virtually  the  same  whether  tne  QI-87  or  the  QI-89  is  used.  Figure  2d  shows  the  expected  percentage 
of  midshipmen  who  will  select  a  technical  major  given  their  score  on  TECH.  This  latter  figure 
indicates  that  the  percentage  of  individuals  selecting  a  technical  major  can  be  increased  by  using 
the  QI-89  as  opposed  to  the  QI-87. 

Adoption  of  the  QI-89  would  increase  the  ability  to  predict  TECH  without  decreasing  the 
ability  to  predict  the  remaining  three  criteria.  In  addition,  the  QI-89  would  provide  the  flexibility 
that  is  necessary  when  evaluating  individual  applicants.  More  specifically,  the  selection  board 
could  utilize  the  ES  scale  in  scholarship  selection  without  being  required  to  assign  it  a  specific 
percentage  of  the  total  selection  points.  Thus,  the  possibility  of  excluding  individuals  simply 
because  they  scored  low  on  the  ES  scale  is  avoided.  In  essence,  an  individual’s  score  on  the  other 
six  predictors  can  be  summed  first,  using  the  weight*  obtained  in  the  regression  procedure,  and 
then  his/her  transformed  ES  score  can  be  considered  by  the  selection  board. 

Equating  QI-89  Scores  to  Previous  QI  Scores 

After  computing  QI-89,  the  score  is  transformed,  via  linear  transformation,  to  a  scale  with  a  M 
and  SD  similar  to  that  of  previous  years.  QI-87  scores  may  range  from  approximately  0  to  300 
whereas  the  unadjusted  QI-89  scores  may  range  from  approximately  0  to  450.  QI-89  scores  are 
transformed  to  the  QI-87  scale  with  the  following  equation: 

Adjusted  QI-89  =  (Raw  QI-89  Score  x  .7786)  -  43.0058 

This  transformation  allows  selection  boards  to  compare  QI  scores  across  years  without  changing 
the  validity  of  QI-89.  That  is,  this  transformation  places  the  QI-89  on  the  same  scale  as  the  QI-  87 
to  provide  consistent  interpretation  of  scores  from  year  to  year. 
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Criterion:  APT 
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SUMMARY  AND  CONCLUSIONS 


1.  All  of  the  multiple-criterion  predictor  composites  except  the  OI  resulted  in  similar  cross- 
validities  when  predicting  GPA  and  NSG.  When  predicting  TECH,  the  cross-validities  for  the  pre¬ 
dictor  composites  varied  a  great  deal.  As  expected,  those  composites  that  included  TECH  as  a  cri¬ 
terion  were,  in  fact,  more  valid  for  predicting  TECH  than  were  the  predictor  composites  without 
TECH.  All  composites  resulted  in  low  cross-validities  for  the  criterion  APT,  including  the  com¬ 
posites  specifically  designed  to  predict  APT. 

2.  Regardless  of  whether  SATM  and  SATV  were  weighted  optimally,  equally,  or  in  a  ratio  of 
70-30,  the  composite  validities  were  similar  when  predicting  GPA  and  NSG.  Composites  that  ei¬ 
ther  optimally  weighted  SATM  and  SATV  or  weighted  SATM  70  percent  and  SATV  30  percent 
were,  however,  more  valid  for  predicting  TECH.  The  latter  findings  were  expected  since  SATM 
was  the  better  predictor  of  TECH  (than  was  SATV). 

3.  The  exclusion  of  the  BQ  in  the  experimental  composite  had  little  impact  on  the  cross-va¬ 
lidities  for  any  criterion. 

4.  Overall,  a  newly  developed  prediction  system  (QI-89)  predicted  GPA,  NSG,  and  APT  as 
well  as  or  slightly  better  than  the  other  predictor  composites.  The  QI-89  can  also  be  used  in  con¬ 
junction  with  the  ES  expectancy  table  to  influence  the  proportion  of  NROTC  scholarship  recipients 
choosing  technical  majors. 


RECOMMENDATIONS 

1 .  CNET  should  use  the  QI-89  and  the  ES  scale  as  aids  in  selecting  future  NROTC  scholarship 
applicants. 

2.  CNET  should  re-evaluate  the  QI-89  as  component  predictors  are  modified  and  improved. 
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TECHNICAL  AND  NON-TECHNICAL  MAJORS 

Note:  The  number  located  to  the  left  of  each  major  is  the  code  given  to  the  particular  major  in  the 
NROTC  database. 

Technical 


Physical  Sciences 

36  Operations  Research/Systems  Analysis  (N);  Operations  Analysis;  Computer 
Operations  Analysis 

37  Meteorology;  Climatology;  Aerology 

38  Chemistry  (other  than  biochemistry);  Wood  Technology 

39  Biochemistry 

40  Ceramics;  Ceramics  Engineering 

42  Metallurgy 

43  Mathematics;  Computer  Science  (Math  Oriented) 

44  Physics 

45  Astronomy 

4C  Physical  Sciences  not  elsewhere  classified 

Engineering  and  Architecture 

47  Civil  Engineering,  General 

48  Agricultural  Engineering 

50  Safety  Engineering;  Fire  Protection  Engineering 

51  Naval  Architecture;  Marine  Engineering 

52  Nuclear  Engineering 

53  Ordnance  Engineering,  General  and  Industrial  (N);  Weapons  Systems,  General  (N); 
Ordnance  Systems  Engineering,  General  (N) 

54  Engineering,  Industrial/Management/Commercial 

55  Chemical  Engineering 

56  Electrical  Engineering,  General 

57  Mechanical  Engineering,  General 

58  Textile  Engineering;  Textile  Technology 

59  Electronics  Engineering 

60  Communications;  Radio  Engineering;  Command  Communications  (N);  Applied 
Communications  (N);  Communications  Engineering  (N)  not  elsewhere  classified 

61  Aeronautical  Engineering 

62  Mining  Engineering 

63  Petroleum  Engineering 

64  Metallurgical  Engineering 

65  Architecture;  Architectural  Engineering;  Landscape  Architecture;  Regional/City 
Planning 

66  Engineering;  Architecture;  Computer  Science  Engineering;  Engineering  and 
Architecture  topics 


Social  Sciences 


90  Statistics 

Technical  Majors  (not  listed  in  above  categories) 

0A  Aeronautics 

OB  Aerospace  Engineering 

OC  Electrical  Power  Engineering 

OD  Materials  Engineering 

OE  Ocean  Engineering 

OF  Petrochemical  Engineering 

OG  Systems  Engineering 

Non-technical 

General 

00  Liberal  Arts 

Agriculture  and  Forestry 

01  Agriculture,  General 

02  Husbandry,  Poultry  and  Animal 

03  Husbandry,  Dairy;  Dairy  Technology 

04  Horticulture 

05  Agronomy;  Soil  Science 

06  Forestry 

07  Range  Science 

08  Agriculture  and  Forestry,  n.e.c.  (Wildlife  Management) 

09  Sciences,  General/Natural/Composite/Basic;  Environmental  Science 


10  Biological  Sciences;  General;  Limnology;  Pre-medicine;  Marine  Biology,  Pre- 
Dental;  Pre-Veterinary 

1 1  Botany,  General 

12  Bacteriology;  Microbiology;  Virology 

13  Animal  Genetics 

14  Physiology 

15  Zoology;  Fish  and  Game  Management;  Ichthyology 

16  Entomology 

17  Parasitology;  Helminthology;  Nematology 

19  Biological  Sciences,  n.e.c. 
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Medical  Sciences 


20  Medicine  and  Surgery 

21  Pharmacy 

22  Public  Health;  Sanitation 

3  Dentistry;  Dental  Surgery 

24  Nursing 

25  Optometry/Physiological  Optics 

26  Veterinary  Science 

27  Pharmacology 

28  Osteopathy 

31  Anatomy 

32  Pathology 

33  Medical  Science,  n.e.c.  (Mortuary  Science;  Medical  Biology,  Medical  Biology, 

Medical  Technology) 

Physical  Sciences 

34  Geology;  Paleontology;  Petrology;  Geological  Engineering 

35  Nautical  Science;  Naval  Science;  Maritime  Academy  Curricula 

Social  Sciences 

67  International  Relations;  International  Law;  Foreign  Affairs;  Foreign  Area  Studies 

68  Political  Science 

69  Public  Administration;  Police  Administration;  Municipal  Government 

70  Industrial  Arts 

71  History;  American  Studies 

72  Industrial  Management 

73  Personnel  Administration  (N)  (may  be  used  when  other  than  Navy  -  Sponsored) 

74  Psychology 

75  Anthropology;  Ethnology 

76  Archeology 

77  Economics 

78  Accounting 

79  Geography 

80  Business  Economics;  Commercial  Education;  Secretarial  Education 

81  Business  Administration;  Advertising;  Commerce;  Foreign  Trade;  Marketing; 
Management;  computer  related  Business  Administration 

82  Banking;  Finance 

83  Merchandising;  Retailing 

84  Physical  Education 

85  Education  (used  when  education  is  a  major  field  of  study;  degrees  based  on  a 
teaching  major  in  a  particular  subject  field  (English,  Biology,  etc.)  are  coded  as 
degrees  in  the  field  itself) 

86  Home  Economics 
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87  Journalism 

88  Law 

89  Library  Science 

91  Social  Work;  Social  Welfare  Administration 

92  Social  Sciences,  n.e.c.  (Sociology;  Criminology,  Pre-Law) 

and  Classics 

93  Advertising  Art;  Dancing;  Drama  (Theater);  Fine  Arts;  Industrial  Design;  Music; 
Printing;  Sculpture 

94  English  Language;  English  Literature 

95  Classical  Languages  and  Literature 

96  Modem  Languages  and  Literature 

97  Theology;  Divinity;  Religion  (professional  ministerial  curricula) 

98  Philosophy;  Religion  (nonsectarian) 

99  Not  a  major  field;  used  for  special  education  reports  prepared  in  BUPERS 
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COMPUTATION  OF  B  WEIGHTS  FOR  A  COMPOSITE  CRITERION 

Computation  of  b  weights  for  an  equation  predicting  a  composite  criterion  requires  five  steps. 
To  illustrate  that  process,  the  steps  required  to  derive  the  b  weight  for  SATV  in  predicting  COMP4 
follow.  First,  unstandardized  b  weights  were  obtained  for  each  of  the  four  individual  criteria.  For 
each  criterion,  all  seven  predictors  were  forced  into  the  multiple-regression  equation.  Second,  the 
b  weight  for  SATV  and  the  standard  deviation  for  the  criterion  were  extracted  from  each  of  the  four 
sets  of  analyses.  Third,  each  b  weight  was  divided  by  the  standard  deviation  of  the  criterion  for 
which  it  had  been  used  as  a  predictor.  Fourth,  the  resulting  quotient  was  multiplied  by  a  CNET- 
supplied  weight:  .30  for  GPA,  .25  for  APT,  .20  for  NSG,  and  .25  for  TECH.  Those  weights 
corresponded  to  the  relative  emphasis  that  CNET  placed  on  each  criterion.  Fifth,  the  products  for 
the  four  criteria  were  then  added  to  produce  the  unstandardized  weight  given  to  SATV  in  the 
prediction  of  the  COMP4  (b^Q  A  j^).  That  calculation  is  depicted  by  the  following  formula: 


bv(G,A,N,T)  =  [(bG  '  SDG}  C30)]  +  [(bA  '  SDA}  (‘25)]  +  [(bN  '  SDN)  (‘20)]  +  l(bT  '  SDT}  ('25)] 


where  b^,  is  the  unstandardized  weight  given  to  SATV  in  the  equation  to  predict  GPA;  b  *  is  the 
unstandardized  weight  given  to  SATV  in  the  equation  to  predict  APT;  bj^  is  the  unstanaardized 
weight  given  to  SATV  in  the  equation  to  predict  NSG;  and  bT  is  the  unstandardized  weight  given 
to  SATV  in  the  equation  to  predict  TECH.  SD^  is  the  standard  deviation  for  GPA;  SPA  is  the  stan¬ 
dard  deviation  for  APT;  SD^  is  the  standard  deviation  for  NSG;  and  SDj  is  the  standard  deviation 
for  TECH.  The  same  steps  were  followed  to  compute  the  un standardized  weights  that  were  as¬ 
signed  to  the  other  six  variables  used  to  predict  the  COMP4.  The  computation  of  the  COMP3  fol¬ 
lowed  all  the  steps  outlined  above,  except  for  step  four.  The  criteria  in  COMP3  were  unweighted; 
therefore,  each  b  weight  was  not  multiplied  by  the  criterion  weights  obtained  from  CNET. 
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